Introduction 

By breaking down the website (Archive.org) into discrete pages, Ul elements and flows we can 
create a series of tests that will aid in evaluation and be short enough to test with users. The 
goal of creating these tests is twofold: to perform initial user testing on V2 and to create a 
roadmapping framework that will enable us to test with users on a regular cadence. 

Website Elements and Pages: 

Elements of the website to be tested 

• Pages + Page Types 

o Homepage 
o Search results page 
o Collection pages 

■ Collection (results) 

■ Collection (about) 
o Item detail pages 

o IA-Meta pages (about, contact, blog, etc) 
o Edit item metadata pages 
o Upload item pages 
o Profile 
o Forum pages 

• Ul Elements 

o Navigation bar 
o Search 

o Advanced Search 
o Media theater 
o Sharing model 
o Item search result previews 
o Collection search result previews 
o Nested collections 

Tasks to be tested 

• Creating a new account 

• Uploading an item 

• Editing an item 

• Downloading an item 

o Different file types 

• Interacting with an item on IA (via media theater) 

o Music 

■ Single song 

■ Album 

■ Albums 
o Movie 

Flippy book 


o 



o Game 
o Arcade game 
o Software 
Searching for: 
o An item 
o A collection 
o A media type 

Navigating to meta-Internet Archive pages 
Sorting search results 

Narrowing a search result by media type or facet 
Using the nav bar to to navigate to a media type, etc 
Favoriting an item and locating favorites 
Reviewing an item 



New User Test 

User Types: Novice - New User | User Testing.com 
Create Account | Library (Meta User Pages) 

Testing goals: 

• Can the user successfully create an account and log-in to Archive.org? 

• Can the user successfully locate their personal library (account profile) on Archive.org? 

• Can the user successfully locate a resource that they would be interested in consuming? 

• Can the user successfully access a resource in the manner they desire? 

1 . Please go to Archive.org 

a. Take a moment to scroll and look around this page without clicking on anything, 
what are your first impressions? 

b. What on the page stands out to you? What are you curious about? 

c. From looking around this page, how do you think you might use this site? 

2. Please create a new account on Archive.org and log-in to your account 

a. How easy or hard was it for you to complete this task on a scale from 1-10, 1 
being very hard, 10 being very easy. 

b. Did anything about creating a new account stand out to you or surprise you? 

3. Please find your Archive.org account profile 

a. How easy or hard was it for you to complete this task on a scale from 1-10, 1 
being very hard, 10 being very easy. 

b. What are your initial impressions of this page? What functionality do were you 
expecting to find here? What would you add or remove to this page if anything? 

4. What would you look for if you were using this site? Please take a few minutes to locate 
a resource that you are interested in, talking aloud throughout your process. 

a. Were you able to find what you were looking for? 

b. How easy or hard was it for you to complete this task on a scale from 1-10, 1 
being very hard, 10 being very easy. 

5. How would you access (read, watch, listen-to, view, play with, download) the resource 
you located in the previous task? 

a. Were the options available to you what you expected? Did it feel like anything 
was missing? 

b. How easy or hard was it for you to complete this task on a scale from 1-10, 1 
being very hard, 10 being very easy. 


Follow-up Questions: 

1 . Have you ever used Archive.org before? If so, have you ever created an account, why or 
why not? 

2. What frustrated you the most? What improvements would have made the process 
easier? 


3. What did you like about the process, if anything? 

4. In what way did this interface match or not match your expectations of Archive.org? 



Upload + Edit 

User Types: Novice - Expert User 


Log-in | Upload Item [ Edit Item 
Testing goals: 

• Can the user successfully log-in? 

• Can the user identify the upload icon on the navigation bar? 

• Can the user successfully upload an item to the Archive? 

• Can the user successfully edit one of their items? 

1 . Please go to Archive.org. 

2. Please log-in to your existing Archive.org account if you have one. If not please log-in 
using these credentials: 

email: carolvn+usertestinq@archive.orq 
password: 12345 

3. Upload a new item 

a. Please open the file ‘cat_picture.jpg’ and take a moment to look at it. What 
characteristics of this image would you want to share with other users of 
Archive.org to aid in their discovery of this image? 

b. Please upload ‘cat_picture.jpg’ to Archive.org. 

c. Once you have uploaded your file, take a moment on the metadata page to scroll 
and look around without clicking on anything, what are your first impressions? 
Describe your emotions upon reaching this screen. 

d. What fields on this form stand out to you? 

e. Please finish uploading your item. 

f. How easy or hard was it for you to complete this task on a scale from 1-10, 1 
being very hard, 10 being very easy. 

4. Please take a moment to look over the page created for the item that you uploaded. 
What are your initial impressions of this page? What, if anything, feels like it’s missing 
for you? What, if anything, are you especially glad to see here? Are any parts of this 
page confusing for you? 

5. Please close your browser window and imagine that you have taken a quick break from 
using Archive.org. If you wanted to go back and edit the item that you added a few 
minutes ago to Archive.org, how would you do so? 

a. Once you have navigated to the editing interface for your item, add the Subject 
Tag ‘Happy Cat’ to this item. 

b. Please save your changes. 

c. How easy or hard was it for you to complete this task on a scale from 1-10, 1 
being very hard, 10 being very easy. 


Follow-up Questions: 

1 . What frustrated you the most? What improvements would have made the tasks you 
performed easier? 

2. What did you like about the process, if anything? 

3. In what way did this interface match or not match your expectations of Archive.org? 



Discovering a Book on Archive.org 

User Types: Novice - Expert User 


Homepage | Navigation [ Search | Search Results | Filtering 1 Sorting 
Testing goals: 

• How do users begin their search for an item on Archive.org? 

o How does the user utilize the navigation bar, homepage or search bar to locate 
an item of interest? 

o Can users identify and utilize sorting and filtering features to aid them in their 
search? 

• Can the user successfully download / access a book item? 

• Can the user successfully identify and navigate file selection in the download area on an 
Item Page? 

1 . Please go to Archive.org 

a. Take a moment to scroll and look around this page without clicking on anything, 
what are your first impressions? 

b. What on the page stands out to you? What are you curious about? 

2. Please show me how you would start looking for a book to read on Archive.org. 

a. Once you’ve located a book that you are interested in, how easy or hard was it 
for you to complete this task on a scale from 1-10, 1 being very hard, 1 0 being 
very easy. 

b. How was this experience different or the same compared to locating a book to 
read on a different library or commerce website (eg. SFPL.org, Amazon.com)? 

c. Was there any part of the interface that was especially helpful to you in your task 
of locating a book to read? Was there anything you felt was missing? 

3. If you wanted to read this book, what steps would you take? 

a. Do you see the option that you would want for accessing this book? 

b. If not, what file type or reading experience 

4. Please download a copy of this book (if book is stream-only use this title) 

a. How easy or hard was it for you to complete this task on a scale from 1-10, 1 
being very hard, 10 being very easy. 

Follow-up Questions: 

1 . How did this interface match with your expectations of using Archive.org, what parts 
were easy to use and behaved as expected? 

2. In what ways did this interface not match your expectations of using Archive.org, what 
parts were hard to use, or weren't quite clear to you? 

3. Are there tasks that you felt you were not successful in completing? 


Collections 

View Testing Sessions 
User Types: New - Expert User 


Search Results | Filtering | Sorting | Collection Page | About Page | Item Page | Item Download 
Testing Goals: 

• How does the user utilize the navigation bar, search or browsing on the homepage to 
find a specific collection? 

• Can the user successfully use the sorting and filtering features on the collections page? 

• Can the user successfully navigate to a collection’s ‘About Page’? 

• Can the user successfully download an item 

• Can the user successfully locate a media-focused collection? 

1. PleasegotoArchive.org 

2. Please go to the ‘American Libraries’ Collection. 

a. How hard or easy was it for you to complete this task on a scale from 1-10, 1 
being very hard, 10 being very easy. 

b. What are your first impressions of this page? What on the page stands out to 
you? What are you curious about? 

3. How do you think the items in this collection are currently ordered? 

a. How would you sort items in this collection by the date they were added to 
Archive.org, with most recently uploaded items first? 

b. How hard or easy was it for you to complete this task on a scale from 1-10, 1 
being very hard, 10 being very easy. 

4. How would you view only items that are in English in this collection? 

a. How hard or easy was it for you to complete this task on a scale from 1-10, 1 
being very hard, 10 being very easy. 

5. How would you find an image of “Children Reading” in this collection? 

a. How hard or easy was it for you to complete this task on a scale from 1-10, 1 
being very hard, 10 being very easy. 

6. If you haven’t already, click on one of the images that you found. 

a. What is your first impression of this page? Does anything feel like it’s missing 
from this page? Are there any aspects of this page that stand out to you? 

b. What collection or collections is this item a part of? 

7. If you wanted to download this image, how would you do so? 

8. Please find and go to one of the Archive’s software collections. 

a. How hard or easy was it for you to complete this task on a scale from 1-10, 1 
being very hard, 10 being very easy. 

9. Where would you go to find out more information about this collection? 

a. What are your first impressions of this page? Do you feel like there is anything 
you would want added to this page? 


b. How hard or easy was it for you to complete this task on a scale from 1-10, 1 
being very hard, 10 being very easy. 

Follow-up Questions: 

1 . How did this interface match with your expectations of using Archive.org, what parts 
were easy to use and behaved as expected? 

2. In what ways did this interface not match your expectations of using Archive.org, what 
parts were hard to use, or weren't quite clear to you? 

3. If you could change one thing about Archive.org, what would you change? 

4. Have you ever created an account on Archive.org, or would you? Why or why not? 



